Thumbnail generating apparatus and thumbnail generating method

ABSTRACT

A thumbnail generating apparatus includes a moving image extracting portion, a reduction and enlargement ratio calculating portion and a reducing and enlarging unit. The moving image extracting portion extracts a moving image corresponding to a period shorter than a reproduction time of moving image based on the number and area of face detection regions obtained from face detection information generated by detecting a face of person included in the moving image. The reduction and enlargement ratio calculating portion integrates the face detection regions at intervals of certain time based on the face detection information of the moving image extracted by the moving image extracting portion, and calculates a reduction and enlargement ratio of the moving image based on the integration result such that a face of person included in the moving image extracted by the moving image extracting portion is largely displayed. The reducing and enlarging unit reduces or enlarges the moving image extracted by the moving image extracting portion based on the reduction and enlargement ratio calculated by the reduction and enlargement ratio calculating portion to generate a moving image thumbnail.

This is a National Phase Application filed under 35 U.S.C. 371 as anational stage of PCT/JP2010/051100, filed Jan. 28, 2010, and claimingthe benefit from Japanese Application No. P2009-073345, filed Mar. 25,2009, the content of which is hereby incorporated by reference in itsentirety.

TECHNICAL FIELD

The present invention relates to a thumbnail generating apparatus and athumbnail generating method for generating a moving image thumbnailaimed at helping a user to perceive contents of a moving image to bereproduced by a reproducing apparatus.

BACKGROUND ART

A video camera with still image thumbnail display function is wellknown. The video camera extracts any one of frames or fieldsconstituting a recorded moving image and reduces it's size, generates itas a still image thumbnail every a recorded moving image or a certainchapter in the recorded moving image, and displays as a thumbnail listscreen these generated still image thumbnails on a liquid crystal panelfor monitor mounted on a main body thereof.

For example, as a technique aimed at helping a user to perceive contentsof an image associated with a still image thumbnail, a patent document 1proposes an image processing apparatus that detects a person's face froma recorded still image, reduces the recorded still image to a certainsize to which the detected person's face is fitted, and generates it asa still image thumbnail.

However, while the technique disclosed in the patent document 1 helps auser to perceive contents of an image associated with a still imagethumbnail in comparison with the conventional still image thumbnaildisplay function, if an image associated with the still image thumbnailis a moving image, there is a case where a user can not perceivecontents of the moving image because the displayed thumbnail is a stillimage.

Meanwhile, there are many cases where an HDD (Hard Disk Drive) recorderor a DVD (Digital Versatile Disk) recorder has motion thumbnail functionfor, when a focus (highlight frame) is matched to one still imagethumbnail on a thumbnail list screen by a user's operation, displaying amoving image thumbnail instead of the still image thumbnail. The motionthumbnail function reproduces a moving image associated with theselected still image thumbnail during a certain period of time,generates a moving image thumbnail in which the reproduced moving imageis reduced to the same size as the still image thumbnail, and displaysthe generated moving image thumbnail on a region of the correspondingstill image thumbnail in a still image thumbnail list screen. A user iseasy to perceive contents of the moving image by watching the movingimage thumbnail displayed by the motion thumbnail function.

CITATION LIST Patent Literature

Patent document 1: Japanese Patent Application Laid-open Publication No.2005-269563

SUMMARY OF INVENTION Technical Problem

It is considered to apply this motion thumbnail function to a videocamera.

However, in a case where the motion thumbnail function is applied to avideo camera, since a liquid crystal panel for monitor mounted on a mainbody thereof has a relatively small screen, a moving image thumbnail tobe displayed on a thumbnail list screen is small. This makes itdifficult for a user to perceive contents of the moving image thumbnail(that is, contents of a moving image associated with the moving imagethumbnail).

The present invention is invented in view of the above problem and hasan object to provide a thumbnail generating apparatus and a thumbnailgenerating method for generating a moving image thumbnail aimed athelping a user to perceive contents of a moving image recorded in arecording and reproducing apparatus and displaying the moving imagethumbnail.

Solution to Problem

In order to achieve the above object, a thumbnail generating apparatusaccording to the present invention has a first feature in which thethumbnail generating apparatus comprises: a moving image extractingportion that, within a first certain period of a moving image, extractseach of images constituting the moving image every a second certainperiod shorter than the first certain period; a reduction andenlargement ratio calculating portion that integrates regions each wherethere is a face of person included in each of the extracted images,determines a face display region such that the face display regionsurrounds the integrated regions each where there is a face of person,and calculates a reduction and enlargement ratio of the moving imagesuch that the determined face display region has substantially the samesize as a certain display region previously set where a moving imagethumbnail is to be displayed; and a moving image thumbnail generatorthat reduces or enlarges the moving image based on the calculatedreduction and enlargement ratio to generate the moving image thumbnail.

In order to achieve the above object, the thumbnail generating apparatusaccording to the present invention has a second feature in which thethumbnail generating apparatus further comprises a thumbnail sectiondetermining portion that determines one or more first certain periodsbased on the number of successive images each of which has a regionwhere there is a face of person, among the images extracted by themoving image extracting portion.

In order to achieve the above object, the thumbnail generating apparatusaccording to the present invention has a third feature in which thereduction and enlargement ratio calculating portion that, if a value ofratio calculated by setting as denominator a value of area of the facedisplay region and setting as numerator a value of total area of theregions each where there is a face of person is smaller than a certainthreshold, the reduction and enlargement ratio calculating portion setsas a representative region a region where there is a face of person,which is located near the center and has the largest area among theregions each where there is a face of person, and calculates thereduction and enlargement ratio such that the representative region hassubstantially the same size as the display region.

In order to achieve the above object, the thumbnail generating apparatusaccording to the present invention has a fourth feature in which thethumbnail generating apparatus further comprises a facial featureinformation storing unit that stores as facial feature information afacial feature for uniquely identifying a face of person, wherein thereduction and enlargement ratio calculating portion determines whetheror not a facial feature in the regions each where there is a face ofperson is the same as or similar to a facial feature represented by thefacial feature information stored in the facial feature informationstoring unit, and if the facial feature is the same as or similar to thefacial feature represented by the facial feature information, thereduction and enlargement ratio calculating portion calculates thereduction and enlargement ratio such that a region where there is a faceof person corresponding to the facial feature being the same as orsimilar to the facial feature represented by the facial featureinformation has substantially the same size as the display region.

In order to achieve the above object, a thumbnail generating methodaccording to the present invention has a first feature in which thethumbnail generating method comprises: a moving image extracting stepof, within a first certain period of a moving image, extracting each ofimages constituting the moving image every a second certain periodshorter than the first certain period; a reduction and enlargement ratiocalculating step of integrating regions each where there is a face ofperson included in each of the extracted images, determining a facedisplay region such that the face display region surrounds theintegrated regions each where there is a face of person, and calculatinga reduction and enlargement ratio of the moving image such that thedetermined face display region has substantially the same size as acertain display region previously set where a moving image thumbnail isto be displayed; and a moving image thumbnail generating step ofreducing or enlarging the moving image based on the calculated reductionand enlargement ratio to generate the moving image thumbnail.

Advantageous Effects of Invention

According to the thumbnail generating apparatus and the thumbnailgenerating method of the present invention, it is possible to generate amoving image thumbnail aimed at helping a user to perceive contents of amoving image recorded in a recording and reproducing apparatus anddisplay the moving image thumbnail.

BRIEF DESCRIPTION OF DRAWINGS

FIG. [1] It is a configuration diagram that illustrates a configurationof a video camera recording on a recording medium an AV stream andstream management information to be supplied to a thumbnail generatingapparatus according to a first exemplary embodiment of the presentinvention.

FIG. [2] It is a configuration diagram that illustrates a configurationof the thumbnail generating apparatus according to the first exemplaryembodiment of the present invention.

FIG. [3] It is an explanatory diagram that illustrates one example of ascreen displayed by a user interface with which the thumbnail generatingapparatus is provided according to the first exemplary embodiment of thepresent invention.

FIG. [4] They are explanatory diagrams that illustrate superposition ofan OSD frame and a moving image reproducing frame on the user interfacewith which the thumbnail generating apparatus is provided according tothe first exemplary embodiment of the present invention. The (a) is anexplanatory diagram that illustrates an order of superposition of theOSD frame and the moving image reproducing frame. The (b) is anexplanatory diagram that illustrates one example of the OSD frame andone example of the moving image reproducing frame. The (c) is anexemplary diagram that illustrates one example of a composite screen inwhich the moving image reproducing frame overlaps with the OSD frame.

FIG. [5] It is an explanatory diagram that illustrates one example ofmoving image thumbnail time information stored in a moving imagethumbnail time storing unit with which the thumbnail generatingapparatus is provided according to the first exemplary embodiment of thepresent invention.

FIG. [6] It is a flowchart that illustrates face detecting processing ina face detector with which the video camera is provided.

FIG. [7] It is a flowchart that illustrates moving image extractingprocessing in the thumbnail generating apparatus according to the firstexemplary embodiment of the present invention.

FIG. [8] It is an explanatory diagram that illustrates processing inwhich a moving image extracting portion of the thumbnail generatingapparatus extracts a “face-having-section lasting for unit divisiontime” according to the first exemplary embodiment of the presentinvention.

FIG. [9] It is a flowchart that illustrates moving image thumbnaildisplaying processing in the thumbnail generating apparatus according tothe first exemplary embodiment of the present invention.

FIG. [10] They are explanatory diagrams that illustrate calculation ofan area “A” of rectangular region, which has an aspect ratio 16:9,surrounding face detection regions by a reduction ratio calculatingportion of a CPU with which the thumbnail generating apparatus isprovided according to the first exemplary embodiment of the presentinvention. The (a) is an explanatory diagram that illustrates facedetection regions during a certain section from a time point “t1” to atime point “t5”. The (b) is an explanatory diagram that illustrates amoving image in a situation where the face detection regions shown in(a) are integrated. The (c) is an explanatory diagram that illustrates amoving image in a situation where the face detection regions dispersetherein. The (d) is an explanatory diagram that illustrates a movingimage in a situation where the face detection regions thicken therein.

FIG. [11] They are explanatory diagrams that illustrate calculation of areduction ratio of a moving image by the reduction ratio calculatingportion with which the thumbnail generating apparatus is providedaccording to the first exemplary embodiment of the present invention.The (a) is an explanatory diagram that illustrates one example of amoving image to be reduced. The (b) is an explanatory diagram thatillustrates one example of the OSD frame.

FIG. [12] They are explanatory diagrams that illustrate calculation of areduction ratio of a moving image by the reduction ratio calculatingportion with which the thumbnail generating apparatus is providedaccording to the first exemplary embodiment of the present invention.The (a) is an explanatory diagram that illustrates one example of amoving image to be reduced. The (b) is an explanatory diagram thatillustrates one example of the OSD frame.

FIG. [13] It is a configuration diagram that illustrates a configurationof a thumbnail generating apparatus according to a second exemplaryembodiment of the present invention.

FIG. [14] It is an explanatory diagram that illustrates one example offacial feature information stored in a facial feature informationstoring unit with which the thumbnail generating apparatus according tothe second exemplary embodiment of the present invention.

FIG. [15] They are explanatory diagrams that illustrate calculation of areduction ratio of a moving image by a reduction ratio calculatingportion with which the thumbnail generating apparatus is providedaccording to the second exemplary embodiment of the present invention.The (a) is an explanatory diagram that illustrates one example of amoving image to be reduced. The (b) is an explanatory diagram thatillustrates one example of the OSD frame.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of the present invention will be described belowwith reference to drawings.

First Exemplary Embodiment

In a first exemplary embodiment of the present invention, we cite avideo camera which includes a face detector that detects a face ofperson based on a captured moving image, and a thumbnail generatingapparatus which generates a moving image thumbnail based on an AV streamand stream management information generated by the video camera andrecorded on a recording medium, as an example of the preset invention.

<Configuration of Video Camera>

FIG. 1 is a configuration diagram that illustrates a configuration of avideo camera which generates an AV stream and stream managementinformation to be used in a thumbnail generating apparatus and recordsthem on a recording medium according to the first exemplary embodimentof the present invention.

The video camera 101 includes an optical lens 111, imaging elements 112,a signal processor 113, an H.264 encoder 114, a multiplexer 115, a facedetector 116, an adder 117, a microphone 121, an A/D converter 122, anAC3 encoder 123 and a CPU 131.

The optical lens 111 has an axis P as an optical axis and focusesentering light.

The imaging elements 112 form an image based on the light focused by theoptical lens 111 and converts into an electrical signal an object imageformed.

By instruction of the CPU 131, the signal processor 113 generates avideo signal based on the electrical signal generated by the imagingelements 112.

By instruction of the CPU 131, the H.264 encoder 114 generates a videostream and GOP (Group of picture) information based on the video signalgenerated by the signal processor 113.

By instruction of the CPU 131, the A/D converter 122 converts soundscollected by the microphone 121 from analog data to digital data.

By instruction of the CPU 131, the AC3 encoder 123 encodes the digitaldata generated by the A/D converter 122 to generate an audio stream andaudio information.

By instruction of the CPU 131, the multiplexer 115 multiplexes the videostream generated by the H.264 encoder 114 and the audio stream generatedby the AC3 encoder 123 to generate an AV stream complying with AVCHDstandard, and records on a recording medium A the AV stream generated.

By instruction of the CPU 131, the face detector 116 carries out facedetecting based on the video signal generated by the signal processor113. More specifically, the face detector 116 samples a moving image atintervals of a predetermined face detection time based on the videosignal, detects a face of person based on the moving image sampled, andsupplies it to the adder 117 as face detection information.

It is noted that if a device or algorism for carrying out processing athigh speed is employed, it is possible to carry out the face detectingprocessing frame by frame. However, if the face detecting processing iscarried out frame by frame, the face detection information has anuneconomically large data size. Accordingly, it is preferable to set theface detection time to about 0.5 seconds, for example. The facedetection information includes the number of face detection regions eachwhere a face of person is detected (one of “0” to “4”, “0” indicatesthat a face of person is not detected), a coordinate (X, Y) of the upperleft of face detection region where a face of person is detected everythe face detection regions each where a face of person is detected in acase where the upper left of moving image is set to a reference point,and the size (width and height) of face detection region where a face ofperson is detected.

Although a moving image consists of frames or fields of successive stillimages and the present invention may be applied to any of them, a stillimage is called as a frame for convenience of the following description.

The face detector 116 also calculates a degree of reliability for a faceof person detected, that is a degree of reliability indicating a degreeof likelihood that a detected object is a face of person, and determineswhether or not the degree of reliability is equal to or more than apredetermined threshold for the degree of reliability.

By instruction of the CPU 131, the adder 117 generates stream managementinformation including the GOP information from the H.264 encoder 114,the audio information from the AC3 encoder 123 and the face detectioninformation from the face detector 116, and records the streammanagement information on the recording medium A.

The CPU 131 carries out central control of the video camera 101. Morespecifically, as described above, the CPU 131 controls the imagingelements 112, the signal processor 113, the H.264 encoder 114, themultiplexer 115, the face detector 116, the adder 117, the A/D converter122 and the AC3 encoder 123.

<Configuration of Thumbnail Generating Apparatus>

FIG. 2 is a configuration diagram that illustrates a configuration of athumbnail generating apparatus according to the first exemplaryembodiment of the present invention.

As shown in FIG. 2, the thumbnail generating apparatus 1 according tothe first exemplary embodiment of the present invention includes ademultiplexer 11, an AC3 decoder 12, a speaker 13, an H.264 decoder 14,a reducing and enlarging unit 15, an adder 16, a user interface 17, anOSD generator 18, a CPU 20 and a moving image thumbnail time storingunit 19.

By instruction of the CPU 20, the demultiplexer 11 reads an AV streamrecorded on the recording medium A and demultiplexer the AV stream togenerate a video stream and an audio stream.

By instruction of the CPU 20, the AC3 decoder 12 decodes the audiostream from the demultiplexer 11 to generate sound data for reproductionand outputs sound from the speaker 13.

By instruction of the CPU 20, the H.264 decoder 14 decodes the videostream from the demultiplexer 11 to generate moving image data forreproduction and supplies the moving image data for reproduction to thereducing and enlarging unit 15.

By instruction of the CPU 20, the reducing and enlarging unit 15 reducesor enlarges the moving image data for reproduction from the H.264decoder 14 to generate reduced or enlarged moving image data forreproduction to be displayed as a moving image thumbnail.

The CPU 20 carries out central control of the thumbnail generatingapparatus 1. More specifically, the CPU 20 controls the demultiplexer11, the AC3 decoder 12, the speaker 13, the H.264 decoder 14, thereducing and enlarging unit 15, the adder 16, the user interface 17 andthe OSD generator 18. The CPU 20 also supplies to the OSD generator 18stream management information read from the recording medium A.

By instruction of the CPU 20, the OSD generator 18 generates ODS imagedata to be used to display an OSD frame based on the stream managementinformation and supplies it to the adder 16.

By instruction of the CPU 20, the adder 16 overlaps the reduced andenlarged moving image data for reproduction from the reducing andenlarging unit 15 with respect to the OSD image data from the OSDgenerator 18 to generate composite image data, and displays thecomposite image data on the user interface 17.

The user interface 17 includes an image display portion composed of anorganic EL (electroluminescence) display, a liquid crystal display orthe like. By instruction of the CPU 20, the user interface 17 displays acomposite screen on the image display portion based on the compositeimage data from the adder 16.

The user interface 17 also includes operation portions such as operationbuttons and a slider composed of touch sensors. When the operationbuttons or the slider is operated by a user, the user interface 17generates an operation signal according to the operation and suppliesthe operation signal to the CPU 20.

The moving image thumbnail time storing unit 19 relates a reproductiontime of an moving image with a reproduction time of a moving imagethumbnail according to the reproduction time of the moving image andstores them. This relation will be described later in detail.

FIG. 3 is an explanatory diagram that illustrates appearance of the userinterface 17 which the thumbnail generating apparatus 1 includesaccording to the first exemplary embodiment of the present invention,and one example of a screen displayed on the image display portion ofthe user interface 17.

As shown in FIG. 3, the user interface 17 includes the image displayportion 17 g, the operation buttons 17 b and the slider 17 c.

An OSD frame including a still image thumbnail generated based on OSDimage data and a moving image thumbnail generated based on moving imagedata for reproduction reduced or enlarged by the reducing and enlargingunit 15 are overlapped each other and displayed on the image displayportion 17 g as a composite image.

A thumbnail display region 17 d, an operation guide display region 17 eand a page display region 17 f are separated from one another anddisplayed on the OSD frame to be displayed on the image display portion17 g.

A still image thumbnail and a moving image thumbnail are displayed onthe thumbnail display region 17 d. This display method will be describedlater in detail.

Icons or the like. for indicating functions assigned to the operationbuttons 17 b to a user are displayed on the operation guide displayregion 17 e.

A page of a still image thumbnail list currently displayed on thethumbnail display region 17 d is displayed on the page display region 17f. For example, when “8/9” is displayed, this indicates that the totalnumber of pages is “9” and the 8-th page is currently displayed.

In an example of FIG. 3, five operation buttons 17 b are arranged. Amongthe five operation buttons 17 b, a leftmost operation button is an OKbutton to be used to generate an operation signal for establishing amoving image thumbnail selected, and a rightmost operation button is amenu button to be used to generate an operation signal for displaying amenu. Fixed functions are not previously assigned to three operationbuttons between these operation buttons, and the CPU 20 assigns variousfunctions according to transition of user operation. The operation guidedisplay region 17 e is used to notify a user of a function assigned toeach operation button. In the example of FIG. 3, function of digestreproduction is assigned to a left operation button, function of zoomreproduction is assigned to a middle operation button, and function oftrash box is assigned to a right operation button.

The slider 17 c is used to slide a highlight frame 17 k which is adisplay region where a moving image thumbnail is displayed, as will bedescribed below. When a user traces the slider 17 c toward the lowerside, the highlight frame 17 k moves from left to right. When a usertraces the slider 17 c toward the upper side, the highlight frame 17 kmoves from right to left.

FIG. 4 is an explanatory diagram that illustrates a method fordisplaying a moving image thumbnail together with still imagethumbnails.

As shown in FIG. 4( a), when a moving image thumbnail is displayedtogether with still image thumbnails, a moving image reproducing frame17 h and an OSD frame 17 a are used.

As shown in FIG. 4( b), the CPU 20 displays the thumbnail display region17 d on the OSD frame 17 a. In the thumbnail display region 17 d, a listof still image thumbnails is displayed. For example, each still imagethumbnail is generated by extracting an I-frame at an arbitrary positionin each moving image, decoding it, and reducing or enlarging the decodedimage.

In FIG. 4( b), when a user operates the slider 17 c, the highlight frame17 k, which has overlapped with the left upper still thumbnail in thethumbnail display region 17 d, moves to select any one of the stillimage thumbnails displayed in the thumbnail display region 17 d.

Then, after any one of the still image thumbnails is selected, when 0.5seconds elapses, the CPU 20 eliminates the still image thumbnailsurrounded by the highlight frame 17 k and provides a transparentdisplay within the highlight frame 17 k.

On the other hand, the CPU 20 reduces or enlarges moving image data forreproduction such that a moving image has the same size as the highlightframe 17 k using the reducing and enlarging unit 15, generates a movingimage thumbnail 17 j so as to be fit to the highlight frame at a displayposition thereof, and provides it as the moving image reproducing frame17 h.

Then, as shown in FIG. 4( c) the CPU 20 overlaps the moving imagereproducing frame 17 h with respect to the back surface of the OSD frame17 a to generate a composite screen. Thereby, a moving image thumbnail17 j on the moving image reproducing frame 17 h is displayed within thehighlight frame 17 k on the OSD frame 17 a within which a transparentdisplay is provided.

FIG. 5 is an explanatory diagram that illustrates one example of movingimage thumbnail time information stored in the moving image thumbnailtime storing unit 19 with which the thumbnail generating apparatus isprovided according to the first exemplary embodiment of the presentinvention.

As shown in FIG. 5, a column name “reproduction time of moving image”(reference number “101”), a column name “reproduction time of movingimage thumbnail” (reference number “102”) and a column name “the numberof reproduction sections of moving image thumbnail” (reference number“103”) are associated with one another and stored as moving imagethumbnail time information. The reproduction time of moving imagethumbnail 102 is calculated by multiplying a unit division time, whichwill be described later, by the number of reproduction sections ofmoving image thumbnail 103.

The CPU 20 includes a moving image extracting portion 20 a, a reductionand enlargement ratio calculating portion 20 b and a thumbnail sectiondetermining portion 20 d in function.

Within a first certain period of moving image, the moving imageextracting portion 20 a extracts each of images constituting an movingimage every a second certain period shorter than the first certainperiod which will be described later.

The thumbnail section determining portion 20 d determines one or morefirst certain periods based on the number of successive images each ofwhich has a region where there is a face of person, among the imagesextracted by the moving image extracting portion 20 a.

The reduction and enlargement ratio calculating portion 20 b integratesone or more regions each where there is a face of person included ineach image extracted, determines a face display region such that theface display region surrounds the one or more regions integrated eachwhere there is a face of person, and calculates a reduction andenlargement ratio of a moving image such that the face display regiondetermined has substantially the same size as a certain display regionpreviously set where a moving image thumbnail is to be displayed.

In a case where a total area of face detection region with respect to anarea of object face region determined is smaller than a certainthreshold, the reduction and enlargement ratio calculating portion 20 bcalculates a reduction and enlargement ratio of a moving image such thata face detection region, which is located near the center of image andhas a large area, has substantially the same size as a moving imagethumbnail as a representative region.

<Working of Face Detector 116 with which Video Camera 101 is Provided>

Next, the face detecting processing carried out by the face detector 116with which the video camera 101 is provided will be described.

FIG. 6 is a flowchart that illustrates the face detecting processing inthe face detector 116 with which the video camera 101 is provided.

As shown in FIG. 6, the face detector 116 samples a frame included in amoving image every a face detection time (second certain period: forexample, 0.5 seconds) based on a video signal, detects a face of personfrom each sampled frame, and generates a tentative number Np of facedetection regions (face regions) each where a face of person is detected(step S201).

Next, the face detector 116 assigns “1” to a counter i for a facedetection region where a face of person is detected as an initial valueand assigns “0” to the effective number M of faces as an initial value(step S202).

Then, the face detector 116 calculates a degree of reliability for thei-th face detection region where a face of person is detected in orderof detection among the tentative number Np of face detection regionseach where a face of person is detected, and determines whether or notthe degree of reliability is equal to or more than a predeterminedthreshold for the degree of reliability (step S203). It is noted that adegree of reliability indicates a degree of likelihood that a detectedobject is a face of person.

In step S203, if the degree of reliability for the i-th face detectionregion is less than the predetermined threshold for the degree ofreliability (NO), the face detector 116 deletes the i-th face detectionregion (step S204).

On the other hand, in step S203, if the degree of reliability for thei-th face detection region is equal to or more than the predeterminedthreshold for the degree of reliability (YES), the face detector 116adds “1” to the effective number M of faces (step S205).

Next, the face detector 116 determines whether or not a value of thecounter i is more than the tentative number Np of face detection regionseach where a face of person is detected (step S206).

In step S206, if the value of the counter i is more than the tentativenumber Np of face detection regions each where a face of person isdetected (YES), the face detector 116 determines whether or not theeffective number M of faces is more than the maximum number Mmax offaces (step S207). It is noted that it is necessary that a providerpreviously calculates a proper value based on actual measurement and theprovider or a user previously sets the proper value to the maximumnumber Mmax of faces (e.g., “4”).

In step S207, if the effective number M of faces is equal to or lessthan the maximum number Mmax of faces (NO), the face detector 116assigns a value of the effective number M of faces to the evaluationnumber Mval of faces (step S208).

On the other hand, in step S207, if the effective number M of faces ismore than the maximum number Mmax of faces (YES), the face detector 116calculates an area of face detection region for each of values up to theeffective number M of faces (step S208).

Next, the face detector 116 sorts the face detection regions for valuesup to the effective number M of faces, in descending order of an area offace detection region calculated in the step S208 (step S209).

Then, the face detector 116 selects Mmax face detection regions startingfrom a face detection region having the largest area, from among theface detection regions sorted in the step S209 (step S210).

The face detector 116 further assigns the value of Mmax to theevaluation number Mval of faces (step S211).

Next, the face detector 116 generates face detection information (stepS212). More specifically, the face detector 116 assigns the value of theevaluation number Mval to the number N of face detection regions eachwhere a face of person is detected, generates face detection informationincluding the number N of face detection regions each where a face ofperson is detected and coordinates (X, Y) and sizes (width, height) of Nface detection regions, and supplies the face detection information tothe adder 117.

According to the face detector 116 of the first exemplary embodiment ofthe present invention with which the video camera 101 is provided, theface detector 116 samples a moving image (frames) at intervals of theface detection time based on the video signal generated by the signalprocessor 113, detects a face of person based on the moving image (eachframe) sampled, and supplies it to the adder 117 as face detectioninformation. As will be described later, this allows the thumbnailgenerating apparatus 1 to generate a moving image thumbnail based on theface detection information.

<Working of Thumbnail Generating Apparatus 1>

Next, working of the thumbnail generating apparatus 1 of the firstexemplary embodiment of the present invention will be described.

The thumbnail generating apparatus 1 of the first exemplary embodimentof the present invention mainly carries out moving image extractingprocessing and moving image thumbnail displaying processing in thisorder. Each processing will be described later in detail.

<Moving Image Extracting Processing>

The moving image extracting processing in the thumbnail generatingapparatus 1 of the first exemplary embodiment of the present inventionwill be described in detail.

FIG. 7 is a flowchart that illustrates the moving image extractingprocessing in the thumbnail generating apparatus 1 according to thefirst exemplary embodiment of the present invention.

First, if the moving image extracting portion 20 a of the CPU 20determines that a predetermined time (e.g., 0.5 seconds) has elapsedafter any one of the still image thumbnails is selected based on anoperation signal from the user interface 17 (step S101), the thumbnailsection determining portion 20 d determines a “face-having-sectionlasting for unit division time (first certain period)” from a successive“face-having-section” lasting for a certain unit division time (here, 5seconds) or more based on the number N of face detection regions (faceregions) each where a face of person is detected included in the facedetection information of the stream management information read from therecording medium A (step S102).

Here, if a unit division time is too long, this lengthens an integrationtime of face regions which will be described later, which prevents amoving image thumbnail from being enlarged and displayed in an effectivemanner. If a unit division time is too short, this finishes images of amoving image thumbnail in a short time, which prevents a user formeasily perceiving contents. Thus, a provider or the like has topreviously calculate a proper value (e.g., 5 seconds) based on actualmeasurement, and the provider, a user or the like has to previously setthe proper value.

FIG. 8 illustrates processing in which the thumbnail section determiningportion 20 d determines a “face-having-section lasting for unit divisiontime” and the moving image extracting portion 20 a extracts the“face-having-section lasting for unit division time”.

As shown in FIG. 8, the stream management information read from therecording medium A includes plural pieces of face detection informationat intervals of the face detection time (0.5 seconds), and each piece offace detection information includes the number of face detection regionsN each where a face of person is detected.

The thumbnail section determining portion 20 d determines as“no-face-having-section” a section where a part in which the number offace detection regions N is “0” successively appears for more than fourparts, that is where a face of person is not detected for more than twoseconds, and determines another section as “face-having-section”.

In the example shown in FIG. 8, a section 205 where a face of person isnot detected for more than two seconds is determined as“no-face-having-section” and a section 206 where a face of person is notdetected for two seconds or less is determined as “face-having-section”.

Then, the moving image extracting portion 20 a extracts aface-having-section lasting for unit division time (207 and 208) fromamong the face-having-section. It is noted that the total number offace-having sections each lasting for unit division time extracted bythe moving image extracting portion 20 a is called the number ofextracted sections S.

Next, the moving image extracting portion 20 a determines whether or notthe number of extracted sections S is less than the number ofreproduction sections of moving image thumbnail 103 based on thereproduction time of moving image included in the stream managementinformation and the moving image thumbnail time information stored inthe moving image thumbnail time storing unit 19 (step S104). Morespecifically, the moving image extracting portion 20 a selects thenumber of reproduction sections of moving image thumbnail 103 accordingto the reproduction time of moving image included in the streammanagement information based on the moving image thumbnail timeinformation stored in the moving image thumbnail time storing unit 19shown in FIG. 5, and determines whether or not the number of extractedsections S is less than the selected number of reproduction sections ofmoving image thumbnail 103.

In step S104, if the number of extracted sections S is less than thenumber of reproduction sections of moving image thumbnail 103 (YES), themoving image extracting portion 20 a determines whether or not there isa “face-having-section” lasting for less than unit division time exceptfor the extracted “face-having-section lasting for unit division time”(step S105).

In step S105, if there is a “face-having-section” lasting for less thanunit division time except for the extracted “face-having-section lastingfor unit division time” (YES), the moving image extracting portion 20 aextracts a “face-having-section” by the deficiency number from thebeginning of moving image, except for the extracted “face-having-sectionlasting for unit division time” (step S106).

Then, the moving image extracting portion 20 a adds the number ofsections extracted in step S106 to the number of extracted sections Sand sets it as the fixed number of extracted sections Sf (step S107).

In step S105, if there is not a “face-having-section” lasting for lessthan unit division time except for the extracted “face-having-sectionlasting for unit division time” (NO), the moving image extractingportion 20 a assigns the number of extracted numbers S to the fixednumber of extracted sections Sf (step S120).

On the other hand, in step S104, if the number of extracted sections Sis equal to or more than the number of reproduction sections of movingimage thumbnail 103 (NO), the moving image extracting portion 20 adetermines whether or not the number of extracted sections S is equal tothe number of reproduction sections of moving image thumbnail 103 (stepS108).

In step S108, the number of extracted sections S is equal to the numberof reproduction sections of moving image thumbnail 103 (YES), the movingimage extracting portion 20 a assigns the number of extracted numbers Sto the fixed number of extracted sections Sf (step S109).

On the other hand, in step S108, the number of extracted sections S isnot equal to the number of reproduction sections of moving imagethumbnail 103 (NO), the moving image extracting portion 20 a calculatesan average value of areas of face detection regions every“face-having-sections each lasting for unit division time” based on thesize (width and height) of each face detection region where a face ofperson is detected included in the face detection information (stepS110).

Next, the moving image extracting portion 20 a sorts the extracted“face-having-sections each lasting for unit division time” in descendingorder of the average value of areas of face detection regions calculatedin step S110 (step S111).

Then, the moving image extracting portion 20 a selects“face-having-sections each lasting for unit division time” starting fromthe largest by the number of reproduction sections of moving imagethumbnail 103, from among the “face-having-sections each lasting forunit division time” sorted in step S111 (step S112).

Next, the moving image extracting portion 20 a assigns the number ofreproduction sections of moving image thumbnail 103 to the fixed numberof extracted sections Sf (step S113).

Thus, since sections each where an average values of areas of facedetection regions is large are preferentially selected from among the“face-having-sections each lasting for unit division time”, sectionseach where a face of person is largely captured are preferentiallyselected.

As described above, by carrying out the moving image extractingprocessing, the thumbnail generating apparatus 1 according to the firstexemplary embodiment of the present invention can extract a moving image(frames) of a reproduction time of moving image thumbnail, which is notlonger than a reproduction time of moving image, from a moving image inwhich a frame where a face of person is not detected in succession for acertain period of time is removed, based on the reproduction time ofmoving image, the number of face detection regions each where a face ofperson is detected at intervals of a face detection time, and the size(width and height) of face detection region where a face of person isdetected.

Thereby, in order to generate a moving image thumbnail corresponding toa still image thumbnail selected, frames constituting proper sectionscan be extracted from among frames constituting an moving image storedin the recording medium A.

<Moving Image Thumbnail Generating and Displaying Processing>

Moving image thumbnail generating and displaying processing of thethumbnail generating apparatus 1 according the first exemplaryembodiment of the present invention will be described in detail.

FIG. 9 is a flowchart that illustrates moving image thumbnail generatingand displaying processing in the thumbnail generating apparatus 1according to the first exemplary embodiment of the present invention.

First, the reduction and enlargement ratio calculating portion 20 b ofthe CPU 20 assigns “0” to a value of counter k for a section as aninitial value (step S301), and assigns “1” to a value of counter j for aface detection region as an initial value (step S302).

Next, the reduction and enlargement ratio calculating portion 20 b ofthe CPU 20 calculates the summation “B” of areas of face detectionregions (face regions) based on the size (width and height) of each facedetection region in the k-th section of “face-having-sections” extractedby the fixed number of extracted sections Sf, included in the facedetection information (step S303).

Further, the reduction and enlargement ratio calculating portion 20 b ofthe CPU 20 calculates an area “A” of rectangular region (face displayregion) having the same aspect ratio (16:9) as a moving imagesurrounding the face detection regions based on the coordinate (X, Y) ofthe upper left of each face detection region and the size (width andheight) of each face detection region in the k-th section, included inthe face detection information (step S304).

FIG. 10 is explanatory diagrams that illustrate calculation of the area“A” of rectangular region, which has the aspect ratio 16:9, surroundingthe face detection regions by the reduction and enlargement ratiocalculating portion 20 b of the CPU 20 with which the thumbnailgenerating apparatus 1 is provided according to the first exemplaryembodiment of the present invention. The (a) illustrates face detectionregions during a certain section from a time point “t1” to a time point“t5”. The (b) is a diagram of frame in a situation where the facedetection regions shown in (a) are integrated. The (c) is a diagram in asituation where the face detection regions disperse in frames. The (d)is a diagram in a situation where the face detection regions thicken inframes.

As shown in FIG. 10( a), the positions of face detection regions 301 to305 moves little by little at the time points “t1” to “t5” as timeadvances.

Then, when these face detection regions 301 to 305 are integrated, anintegrated face detection region 401 is obtained as shown in FIG. 10(b).

Thus, the reduction and enlargement ratio calculating portion 20 bcalculates an area “A” of rectangular region 402, which has the aspectratio (16:9), surrounding the integrated face detection region 401.

Since the reduction and enlargement ratio calculating portion 20 bcalculates an area “A” of rectangular region as described above, an area“A” of rectangular region 403 is large as shown in FIG. 10( c) in a casewhere the face detection regions 301 to 305 disperse in frames, and anarea “A” of rectangular region 404 is small as shown in FIG. 10( d) in acase where the face detection regions 301 to 305 thicken in frames.

Next, the reduction and enlargement ratio calculating portion 20 b ofthe CPU 20 determines whether or not the summation “B” of areas of facedetection regions calculated in step S303 with respect to the area “A”of rectangular region calculated in step S304 is less than apredetermined threshold “R” (step S305).

Here, if the threshold “R” is too small, a moving image thumbnail cannot be effectively enlarged and displayed because even if an area “A” ofrectangular region is large, a reduction and enlargement ratio of amoving image is calculated such that the area of rectangular region hasthe same area as a moving image thumbnail as will be described below.Also, if the threshold “R” is too large, a moving image thumbnail cannot be effectively enlarged and displayed because even if a user wantsto enlarge respective face detection regions for a reason that a area“A” of rectangular region is small, a reduction and enlargement ratio ofa moving image is calculated such that only one face detection regionwhere an evaluation score “P” is the highest is enlarged.

In step S305, if the summation “B” of areas of face detection regionswith respect to the area “A” of rectangular region is equal to or morethan a predetermined threshold “R” (NO), the reduction and enlargementratio calculating portion 20 b calculates a reduction and enlargementratio of a moving image such that the size (width and height) ofrectangular region is the same as the size (width and height) of movingimage thumbnail (step S306).

FIG. 11 is explanatory diagrams that illustrate calculation of areduction and enlargement ratio of a moving image by the reduction andenlargement ratio calculating portion 20 b with which the thumbnailgenerating apparatus 1 is provided according to the first exemplaryembodiment of the present invention. The (a) illustrates one example ofa frame of moving image to be reduced or enlarged. The (b) illustratesone example of the OSD frame 17 a.

As shown in FIG. 11( a), the size of frame of moving image is X*Y(aspect ratio 16:9) and the size of rectangular region 501 is X2*Y2(aspect ratio 16:9). As shown in FIG. 11( b), the size of moving imagethumbnail in the OSD frame 17 a, that is the size of highlight frame 17k is X1*Y1 (aspect ratio 16:9).

In this case, the reduction and enlargement ratio calculating portion 20b calculates a reduction and enlargement ratio of the moving image suchthat the size of rectangular region 501 is the same as the size ofhighlight frame 17 k. Namely, if we assume that the reduction andenlargement ratio is “Q”, the reduction and enlargement ratio “Q” iscalculated according to the following equation 1.Q=(X1/X)*(X1/X2)  (equation 1)

It is noted that a value of “X2” in (X1/X2) represents not the size of“X2” in a frame of an original moving image shown in FIG. 11( a) butinstead the size of “X2” in a frame of a moving image adjusted to amoving image thumbnail shown in FIG. 11( b). Thus, since the reductionand enlargement ratio calculating portion 20 b calculates a reductionand enlargement ratio of a moving image such that the size ofrectangular region 501 is the same as the size of highlight frame 17 k,if (X1/X2) is larger than (X1/X), a moving image looks like it isenlarged and displayed as a moving image thumbnail. However, an image inthe moving image thumbnail is actually reduced or enlarged from theoriginal moving image and then displayed.

On the other hand, in step S305 of FIG. 9, if the summation “B” of areasof face detection regions with respect to the area “A” of rectangularregion is less than the predetermined threshold “R” (YES), the reductionand enlargement ratio calculating portion 20 b calculates an evaluationscore “P(j)” of the j-th face detection region in N face detectionregions each where a face of person is detected in the k-th sectionaccording to the following equation 2 (step S307).P(j)=area of the j-th face detection region−(length between the centerof screen and the center of the j-th face detectionregion)2*0.2  (equation 2)

Then, the reduction and enlargement ratio calculating portion 20 b adds“1” to the value of counter “j” (step S308), and then determines whetheror not the value of counter “j” is larger than the number “N” of facedetection regions each where a face of person is detected (step S309).

In step S309, if the value of counter “j” is larger than the number “N”of face detection regions each where a face of person is detected (YES),the reduction and enlargement ratio calculating portion 20 b selects asa representative region a face detection region which is near the centerof screen and has a large area, that is a face detection region wherethe value of evaluation score “P” is the largest, from among the N facedetection regions each where a face of person is detected (step S310).

Next, the reduction and enlargement ratio calculating portion 20 bcalculates a reduction and enlargement ratio of moving image such thatthe size (width and height) of representative region selected in stepS310 is substantially the same as the size (width and height) of movingimage thumbnail (step S311).

FIG. 12 is explanatory diagrams that illustrate calculation of areduction and enlargement ratio of a moving image by the reduction andenlargement ratio calculating portion 20 b with which the thumbnailgenerating apparatus 1 is provided according to the first exemplaryembodiment of the present invention. The (a) illustrates one example ofa frame of a moving image to be reduced or enlarged. The (b) illustratesone example of the OSD frame 17 a.

As shown in FIG. 12( a), the size of frame of moving image is X*Y(aspect ratio 16:9) and the size of representative region 502 is X3*Y3(aspect ratio 16:9). As shown in FIG. 12( b), the size of moving imagethumbnail in the OSD frame 17 a, that is the size of highlight frame 17k is X1*Y1 (aspect ratio 16:9).

In this case, the reduction and enlargement ratio calculating portion 20b calculates a reduction and enlargement ratio of the moving image suchthat the size of representative region 502 is the same as the size ofhighlight frame 17 k. Namely, if we assume that the reduction andenlargement ratio is “Q”, the reduction and enlargement ratio “Q” iscalculated according to the following equation 1 in which “X2” and “Y2”are respectively replaced by “X3” and “Y3”.

Thus, since the reduction and enlargement ratio calculating portion 20 bcalculates a reduction and enlargement ratio of the moving image suchthat the size (width and height) of representative region selected instep S310 is substantially the same as the size (width and height) ofmoving image thumbnail, it is easy to enlarge a face of person, whichhelps a user to perceive contents of a moving image easily.

Next, the reducing and enlarging unit 15 reduces or enlarges a movingimage based on a reduction and enlargement ratio calculated by thereduction and enlargement ratio calculating portion 20 b to generate amoving image thumbnail (step S312).

Thereby, the adder 16 to which image data for reproduction including themoving image thumbnail generated is supplied merges two frames based onOSD image data such that the OSD frame is overlapped with the movingimage reproducing frame, and then displays the composite screen on theuser interface 17.

Next, the reduction and enlargement ratio calculating portion 20 b ofthe CPU 20 adds “1” to the value of counter k (step S313), determineswhether or not the counter k is equal to or more than the fixed numberof extracted sections Sf (step S314), and repeats the processing ofsteps S303 to S314 until the counter k is equal to or more than thefixed number of extracted sections Sf.

As described above, the thumbnail generating apparatus 1 according tothe first exemplary embodiment of the present invention can generate anddisplay a moving image thumbnail which helps a user to perceive contentsof a moving image easily.

Although the first exemplary embodiment of the present inventiondescribes as one example the thumbnail generating apparatus to whichface detection information regarding a face of person detected issupplies from the video camera 101 via the medium A, generating a movingimage thumbnail based on the face detection information supplied, it isnot limited to this. The thumbnail generating apparatus may be connectedto the video camera 101 via a network, and the thumbnail generatingapparatus to which face detection information regarding a face of persondetected from the video camera 101 via the network is supplied maygenerate a moving image thumbnail based on the face detectioninformation supplied.

Second Exemplary Embodiment

The first exemplary embodiment describes as one example the thumbnailgenerating apparatus to which face detection information regarding aface of person detected is supplies via the recording medium A from thevideo camera 101 provided with the face detector detecting a face ofperson based on a moving image captured, reducing or enlarging themoving image to generate a moving image thumbnail based on the facedetection information supplied. In addition, the thumbnail generatingapparatus may reduce or enlarge the moving image to generate a movingimage thumbnail based on facial feature information stored.

The second exemplary embodiment describes as one example a thumbnailgenerating apparatus to which face detection information regarding aface of person detected is supplies via the recording medium A from thevideo camera 101 provided with the face detector detecting a face ofperson based on a moving image captured, reducing or enlarging themoving image to generate a moving image thumbnail based on the facedetection information supplied and facial feature information.

<Configuration of Video Camera 101>

Since a configuration of video camera 101 which records on the recordingmedium A an AV stream and stream management information to be suppliedto the thumbnail generating apparatus according to the second exemplaryembodiment of the present invention is the same as the configuration ofthe video camera 101 which records on the recording medium A an AVstream and stream management information to be supplied to the thumbnailgenerating apparatus 1 according to the first exemplary embodiment ofthe present invention, the explanation is omitted.

<Configuration of Thumbnail Generating Apparatus>

FIG. 13 is a configuration diagram that illustrates a configuration ofthe thumbnail generating apparatus 1A according to the second exemplaryembodiment of the present invention.

As shown in FIG. 13, the thumbnail generating apparatus 1A according tothe second exemplary embodiment of the present invention includes thedemultiplexer 11, the AC3 decoder 12, the speaker 13, the H.264 decoder14, the reducing and enlarging unit 15, the adder 16, the user interface17, the OSD generator 18, a CPU 20A, the moving image thumbnail timestoring unit 19 and a facial feature information storing unit 21.

Since the demultiplexer 11, the AC3 decoder 12, the speaker 13, theH.264 decoder 14, the reducing and enlarging unit 15, the adder 16, theuser interface 17, the OSD generator 18, and the moving image thumbnailtime storing unit 19 are the same as those to which the same referencenumbers are assigned in the thumbnail generating apparatus 1 accordingto the first exemplary embodiment of the present invention, theexplanation is omitted.

The facial feature information storing unit 21 associates an ID ofperson for uniquely identifying the person with feature information ofthe person's face, and then stores it as facial feature information.

FIG. 14 illustrates one example of facial feature information stored inthe facial feature information storing unit 21 with which the thumbnailgenerating apparatus 1A according to the second exemplary embodiment ofthe present invention is provided.

As shown in FIG. 14, a column name “ID of person” (reference number 601)and a column name “feature information” (reference number 602) areassociated with each other and stored as facial feature information. Itis noted that the facial feature information is information forrepresenting a facial feature capable of uniquely identifying a face ofperson such as a size, a shape and the like of eyes, a nose, a mouth andthe like.

The CPU 20A carries out central control of the thumbnail generatingapparatus 1A. The CPU 20A includes a moving image extracting portion 20a and a reduction and enlargement ratio calculating portion 20 c infunction.

The moving image extracting portion 20 a extracts frames of a movingimage during a period which does not exceed a reproduction time ofmoving image thumbnail from among frames of the moving image in whichframes during a period when a face is not successively detected for apredetermined time are eliminated, based on a reproduction time ofmoving image and the number of facial images detected at intervals of aface detection time.

If face detection information including facial feature information issupplied from the video camera 101, the reduction and enlargement ratiocalculating portion 20 c determines whether or not the facial featureinformation included in the face detection information supplied is thesame as or similar to facial feature information stored in the facialfeature information storing unit 21. If the facial feature informationis the same as or similar to it, the reduction and enlargement ratiocalculating portion 20 c calculates a reduction and enlargement ratio ofa moving image such that face detection region corresponding to thefacial feature information being the same as or similar to it ismaximized.

More specifically, the reduction and enlargement ratio calculatingportion 20 c selects as a registered face region a face detection regioncorresponding to the facial feature information being the same as orsimilar to it, and then calculates a reduction and enlargement ratio ofthe moving image such that the size of registered face region selectedis substantially the same as the size of moving image thumbnail.

FIG. 15 is explanatory diagrams that illustrate calculation of areduction and enlargement ratio of a moving image by the reduction andenlargement ratio calculating portion 20 c with which the thumbnailgenerating apparatus 1A is provided according to the second exemplaryembodiment of the present invention. The (a) illustrates one example ofa moving image to be reduced or enlarged. The (b) illustrates oneexample of the OSD frame.

As shown in FIG. 15( a), the size of frame of moving image is X*Y(aspect ratio 16:9) and the size of registered face region 601 is X4*Y4(aspect ratio 16:9). As shown in FIG. 15( b), the size of moving imagethumbnail in the OSD frame 17 a, that is the size of highlight frame 17k is X1*Y1 (aspect ratio 16:9).

In this case, the reduction and enlargement ratio calculating portion 20b calculates a reduction and enlargement ratio of the moving image suchthat the size of registered face region 601 is the same as the size ofhighlight frame 17 k. Namely, if we assume that the reduction andenlargement ratio is “Q”, the reduction and enlargement ratio “Q” iscalculated according to the following equation 1 in which “X2” and “Y2”are respectively replaced by “X4” and “Y4”.

Thus, since the reduction and enlargement ratio calculating portion 20 ccalculates a reduction and enlargement ratio of the moving image suchthat the size of registered face region selected is substantially thesame as the size of moving image thumbnail, it is easy to enlarge a faceof person recorded. Thereby, it is easy for a user to watch a face imageof a desired person's face, which helps the user to perceive contents ofa moving image easily.

As described above, since the thumbnail generating apparatus 1Aaccording to the second exemplary embodiment of the present inventionincludes the facial feature information storing unit 21 and thereduction and enlargement ratio calculating portion 20 c, it is possibleto preferentially enlarge one or more persons previously registered anddisplay it as a moving image thumbnail, in addition to the effect of thethumbnail generating apparatus 1 according to the first exemplaryembodiment of the present invention.

In the second exemplary embodiment, in a case where a face which is thesame as a facial feature previously stored is detected from among facesbeing detected, if only the face which is the same as it is set asprocessing object, a face of only user's children can be enlarged anddisplayed as a moving image thumbnail in a moving image in which theuser's children is captured together with another person's children, forexample.

Reference Signs List

-   1, 1A . . . thumbnail generating apparatus-   11 . . . demultiplexer-   12 . . . AC3 decoder-   13 . . . speaker-   14 . . . H.264 decoder-   15 . . . reducing and enlarging unit-   16 . . . adder-   17 . . . user interface-   17 a . . . image display region-   17 b . . . operation buttons-   17 c . . . slider-   17 d . . . thumbnail display region-   17 e . . . operation guide display region-   17 f . . . page display region-   17 g . . . image display region-   17 k . . . highlight frame-   17 h . . . moving image reproducing frame-   17 j . . . moving image thumbnail-   18 . . . OSD generator-   19 . . . moving image thumbnail storing time-   20, 20A, 131 . . . CPU-   20 a . . . moving image extracting portion-   20 b, 20 c . . . reduction and enlargement ratio calculating portion-   20 d . . . thumbnail section determining portion-   21 . . . facial feature information storing unit-   101 . . . video camera-   111 . . . optical lens-   112 . . . imaging elements-   113 . . . signal processor-   114 . . . H.264 encoder-   115 . . . multiplexer-   116 . . . face detector-   117 . . . adder-   121 . . . microphone-   122 . . . A/D converter-   123 . . . AC3 encoder

The invention claimed is:
 1. A thumbnail generating apparatuscomprising: a moving image extracting portion that, within a firstcertain period of a moving image, extracts each of the imagesconstituting the moving image every a second certain period shorter thanthe first certain period; a reduction and enlargement ratio calculatingportion that integrates each region where there is a face of personincluded in the images extracted within the first certain period,determines a face display region to be used in common with the firstcertain period and having the same aspect ratio as the moving imageextracted by the moving image extracting portion such that the facedisplay region surrounds each of the integrated regions where there is aface of a person, and calculates a reduction and enlargement ratio ofthe moving image such that the determined face display region hassubstantially the same size as a certain display region having the sameaspect ratio as the moving image extracted by the moving imageextracting portion and previously set where a moving image thumbnail isto be displayed; and a moving image thumbnail generator that reduces orenlarges the moving image based on the calculated reduction andenlargement ratio to generate the moving image thumbnail.
 2. Thethumbnail generating apparatus according to claim 1, further comprisinga thumbnail section determining portion that determines one or morefirst certain periods based on the number of successive images each ofwhich has a region where there is a face of person, among the imagesextracted by the moving image extracting portion.
 3. The thumbnailgenerating apparatus according to claim 2, wherein the reduction andenlargement ratio calculating portion that, if a value of ratiocalculated by setting as denominator a value of area of the face displayregion and setting as numerator a value of total area of each of theregions where there is a face of person is smaller than a certainthreshold, the reduction and enlargement ratio calculating portion setsas a representative region a region where there is a face of person,which is located near the center and has the largest area among each ofthe regions where there is a face of person, and calculates thereduction and enlargement ratio such that the representative region hassubstantially the same size as the display region.
 4. The thumbnailgenerating apparatus according to claim 2, further comprising a facialfeature information storing unit that stores as facial featureinformation a facial feature for uniquely identifying a face of person,wherein the reduction and enlargement ratio calculating portiondetermines whether or not a facial feature in each of the regions wherethere is a face of person is the same as or similar to a facial featurerepresented by the facial feature information stored in the facialfeature information storing unit, and if the facial feature is the sameas or similar to the facial feature represented by the facial featureinformation, the reduction and enlargement ratio calculating portioncalculates the reduction and enlargement ratio such that a region wherethere is a face of person corresponding to the facial feature being thesame as or similar to the facial feature represented by the facialfeature information has substantially the same size as the displayregion.
 5. The thumbnail generating apparatus according to claim 1,wherein the reduction and enlargement ratio calculating portion that, ifa value of ratio calculated by setting as denominator a value of area ofthe face display region and setting as numerator a value of total areaof each of the regions where there is a face of person is smaller than acertain threshold, the reduction and enlargement ratio calculatingportion sets as a representative region a region where there is a faceof person, which is located near the center and has the largest areaamong each of the regions where there is a face of person, andcalculates the reduction and enlargement ratio such that therepresentative region has substantially the same size as the displayregion.
 6. The thumbnail generating apparatus according to claim 1,further comprising a facial feature information storing unit that storesas facial feature information a facial feature for uniquely identifyinga face of person, wherein the reduction and enlargement ratiocalculating portion determines whether or not a facial feature in eachof the regions where there is a face of person is the same as or similarto a facial feature represented by the facial feature information storedin the facial feature information storing unit, and if the facialfeature is the same as or similar to the facial feature represented bythe facial feature information, the reduction and enlargement ratiocalculating portion calculates the reduction and enlargement ratio suchthat a region where there is a face of person corresponding to thefacial feature being the same as or similar to the facial featurerepresented by the facial feature information has substantially the samesize as the display region.
 7. A thumbnail generating method comprising:a moving image extracting step of, within a first certain period of amoving image, extracting from non-transitory stored image data each ofthe images constituting the moving image every a second certain periodshorter than the first certain period; a reduction and enlargement ratiocalculating step of integrating each region where there is a face ofperson included in the images extracted within the first certain period,determining a face display region to be used in common with the firstcertain period and having the same aspect ratio as the moving imageextracted by the moving image extracting portion such that the facedisplay region surrounds each of the integrated regions where there is aface of person, and calculating a reduction and enlargement ratio of themoving image such that the determined face display region hassubstantially the same size as a certain display region having the sameaspect ratio as the moving image extracted by the moving imageextracting portion and previously set where a moving image thumbnail isto be displayed; and a moving image thumbnail generating step ofreducing or enlarging the moving image based on the calculated reductionand enlargement ratio to generate the moving image thumbnail.